Determining maximum exposure time to limit motion blur during image capture

ABSTRACT

A method, apparatus and image capture device for limiting motion blur during capture of an image predetermines a relationship between movement of the image capture device, time and blur extent. A rate of movement of the image capture device is obtained and used in conjunction with the determined relationship and a blur extent limit as a basis for obtaining a maximum exposure time for the image capture device in order to limit blur extent. On the basis of the maximum exposure time obtained and a required image brightness, the image capture device is configured, and then an image is captured by the image capture device. Once the image has been captured, due to the possibility of under-exposure from having limited the exposure time, the image is processed as required to increase its intensity.

FIELD OF THE INVENTION

The present invention relates generally to image capture and morespecifically, to a method, apparatus and image capture device forlimiting motion blur during capture of an image.

BACKGROUND OF THE INVENTION

Image capture devices such as digital cameras have become very populardue in part to a reduction in production costs, increase in overallquality, and particularly because camera functionality is being embeddedinto other electronic consumer devices such as cellular telephones andpersonal digital assistants (PDAs).

Image blur due to unsteady hand movement or the like during an imagecapture operation is often difficult to avoid for the inexperiencedphotographer or a user with an unsteady hand. Blur in an image isfrustrating, as it detracts from the appeal of the image. With standardauto-exposure functions in image capture devices, low-light conditionsare compensated for by lowering the shutter speed of the image capturedevices, thereby increasing the exposure time in order to capture abright enough image. This increase in exposure time increases thelikelihood that movement will occur during the exposure time, thusincreasing the likelihood of blur in the captured image. The same blursituation may arise if high-speed image capture device movement occursduring image capture, where the exposure time is normal.

Methods for post-processing a captured image to remove blur are known.These methods typically involve measuring the extent of blur in thecaptured image, and correcting for the measured blur. However, undervery low-light and/or very high speed conditions that cause extensiveblur, the extent of the blur can be so great that such post-processingcannot restore the image.

U.S. Pat. No. 6,778,210 to Sugahara et al. discloses a system thatlimits exposure time in a digital camera to keep blur within allowableor undetectable limits. The exposure time is adjusted based on thephysical dimensions of the camera, on the basis that image blur relativeto the imaging frame is inversely proportional to frame size and focallength. According to the Sugahara et al. method, in the event that thetotal manual or photometrically set exposure time exceeds the exposuretime limit for acceptable blur, the camera is set to take a series oflimited exposure images (their respective exposure times adding up tothe total exposure time) by controlling the accumulation times of theimage pickups. The images are subsequently processed for respectivemotion, correlated, and the motion is corrected. Each resultant imagecontains acceptable levels of blur, and results in an output imagehaving an acceptable level of blur. One disclosed system recognizesreduced signal levels due to short exposure time and employs a velocitysensor instead of a motion correlation algorithm in order to quantifythe motion.

U.S. Patent Application Publication No. 2004/0239771 to Habe discloses amethod for controlling image blur quantity by adjusting the ISO speedsetting of a digital camera. Prior to image capture, the camera in ablur prevention mode engages a vibration detector to, along with a knownshutter speed and focal length, calculate a blur quantity. If thecalculated blur quantity is greater than a predefined limit, the ISOspeed of the camera is shifted to higher sensitivity by an extent thatit is based on the ratio of calculated blur to the predetermined blurlimit. In doing so, the aperture (i.e., field depth) is not changed. Themethod retains image quality by basing the adjustment of ISO speed onthe image blur quantity, rather than on the shutter speed. While the ISOspeed may be raised, it may also be lowered to gain higher-qualityimages having increased blur that it is within acceptable limits.

While blur correction algorithms, and blur reduction and limitingmethods are known, improvements are of course desirable. It is thereforean object to provide a method, apparatus and image capture device forlimiting motion blur during image capture.

SUMMARY OF THE INVENTION

Accordingly, in one aspect there is provided a method of determining amaximum exposure time to limit motion blur during capture of an imagewith an image capture device, comprising:

determining extents of image blur that result from respectivecombinations of movement rate and exposure time of the image capturedevice;

in accordance with a predetermined blur extent limit and thecombinations, deriving a relationship between movement rate and exposuretime of the image capture device;

measuring a movement rate of the image capture device; and

based on the measured movement rate and the relationship, obtaining themaximum exposure time.

In one embodiment, the determining comprises capturing a plurality ofimages using respective combinations of movement rate and exposure timeof the image capture device and then measuring a blur extent in each ofthe captured images.

In another embodiment, the determining comprises defining a model ofcorrelation between movement of the image capture device relative to asubject and resultant blur extent in pixels. Then blur extents resultingfrom rates of movement of the image capture device over time arecalculated using the model.

According to another aspect there is provided a method of capturing animage with an image capture device, comprising:

determining a movement rate of the image capture device;

on the basis of the movement rate and a blur extent limit, obtaining amaximum exposure time from a predetermined relationship between movementrate and exposure time for the image capture device; and

exposing an image sensor of the image capture device to capture animage, the exposing being limited in duration by the maximum exposuretime.

In one embodiment, in the event that a measure of intensity of thecaptured image is less than a predetermined threshold, the intensityvalue of at least a portion of pixels in the captured image isautomatically increased.

In a related embodiment, the amount of intensity increasing is based ona comparison between a measure of intensity of the captured image and asecond measure of intensity that would be captured using an automaticexposure function of the image capture device in order to retainsufficient exposure.

According to another aspect there is provided an apparatus fordetermining a maximum exposure time to limit motion blur during captureof an image with an image capture device, comprising:

a processor determining extents of image blur that result fromrespective combinations of movement rate and exposure time of the imagecapture device and, in accordance with a predetermined blur extent limitand the determined combinations, deriving a relationship betweenmovement rate and exposure time of the image capture device;

a storage device storing the relationship;

a gyro measuring a movement rate of the image capture device; and

a lookup module determining from the relationship stored on the storagedevice the maximum exposure time-based on the measured movement and thepredetermined blur extent limit.

In one embodiment, the processor performs the determining by defining amodel of correlation between movement of the image capture devicerelative to a subject and resultant blur extent in pixels and calculatesblur extents resulting from rates of movement of the image capturedevice over time using the model.

The model of correlation may be based on a simple lens model.

In accordance with still another aspect, there is provided an apparatusfor controlling an image capture device during capture of an image,comprising:

a gyro determining a movement rate of the image capture device;

a processor for, on the basis of the movement rate and a blur extentlimit, obtaining a maximum exposure time from a predeterminedrelationship between movement rate and exposure time for the imagecapture device and controlling the image capture device to expose animage sensor of the image capture device to capture an image, theexposing being limited in duration by the maximum exposure time.

In one embodiment, the apparatus comprises a post-processing module for,in the event that a measure of intensity of the captured image is lessthan a predetermined threshold, automatically increasing the intensityvalue of at least a portion of pixels in the captured image.

In accordance with yet another aspect, there is provided an imagecapture device, comprising:

an image sensor for capturing an image when exposed to light;

a gyro determining a movement rate of the image capture device;

a processor for, on the basis of the movement rate and a blur extentlimit, obtaining a maximum exposure time from a predeterminedrelationship between movement rate and exposure time for the imagecapture device and controlling the image capture device to expose theimage sensor of the image capture device to capture an image, theexposing being limited in duration by the maximum exposure time.

The subject method, apparatus and image capture device described hereinprovide numerous advantages. In particular, an image capture deviceemploying the method is able to limit exposure time of its sensor tocapture an image with an extent of blur due to image capture devicemotion that is tolerable. According to some embodiments, the imagecapture device is able to compensate for reduced exposure time to obtainimages that are sufficiently bright by increasing sensitivity of animage capture sensor, increasing the aperture, or by employing animage-brightening process after image capture. According to stillfurther embodiments, the exposure time limits for various combinationsof angular velocity and acceptable blur extents are pre-calculated andstored in lookup tables, providing extremely fast operation at the timeof image capture. In one advantageous embodiment, exposure time of theimage capture device is limited only if the standard intensity-basedautomatic-exposure exposure time setting of the image capture device isgreater than that which would yield the maximum acceptable blur extent.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will now be described more fully with reference to theaccompanying drawings, in which:

FIG. 1 is a block diagram of a motion blur limiting apparatus;

FIG. 2 is a block diagram of an image capture device incorporating themotion blur limiting apparatus of FIG. 1;

FIG. 3 is a flowchart showing the steps performed to limit blur extentin an image captured using the image capture device of FIG. 2;

FIG. 4 is a flowchart showing steps for determining a relationshipbetween movement of the image capture device, time and blur extent;

FIG. 5 is a schematic diagram showing a perspective image capture devicemodel that is based on the simple lens concept;

FIG. 6 is a diagram showing a field of motion vectors representingmovement about the X-axis;

FIG. 7 is a diagram showing a field of motion vectors representingmovement about the Y-axis;

FIG. 8 is a diagram showing a field of motion vectors representingmovement about the Z-axis;

FIG. 9 is a flowchart showing steps for obtaining a rate of movement ofthe image capture device using gyros;

FIG. 10 is a flowchart showing steps for obtaining a maximum exposuretime based on the relationship, the rate of movement of the imagecapture device and a blur extent limit;

FIG. 11 is a flowchart showing steps for configuring the image capturedevice based on the maximum exposure time and required image intensity;

FIG. 12 is a diagram showing both linear and logarithmic image intensitysaturation curves;

FIG. 13 is a flowchart showing steps for processing a captured image toincrease intensity;

FIG. 14 is a flowchart showing alternative steps for determining arelationship between movement of the image capture device, time and blurextent;

FIG. 15 is an array of test images taken at various combinations ofexposure time and angular velocity of the image capture device usedduring the alternative steps of FIG. 12;

FIG. 16 is a graph showing a relationship between exposure time and blurextent measured in the images of FIG. 13;

FIG. 17 is a graph showing relationships between exposure time andangular velocity for various acceptable blur extents;

FIG. 18 is a flowchart showing alternative steps for configuring theimage capture device based on the maximum exposure time and requiredimage brightness settings;

FIG. 19 is a graph of image intensity vs. exposure time illustrating thealternative steps of FIG. 16;

FIG. 20 is a flowchart showing alternative steps for processing thecaptured image to increase intensity;

FIG. 21 is an array of images and corresponding pixel intensityhistograms illustrating the alternative steps of FIG. 20; and

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following description, a method, apparatus and image capturedevice for limiting motion blur during capture of an image is provided.During the method, a relationship is pre-determined between movement ofthe image capture device, time and blur extent. A rate of movement ofthe image capture device is obtained and used in conjunction with thedetermined relationship and a blur extent limit as a basis for obtaininga maximum exposure time for the image capture device in order to limitblur extent. On the basis of the maximum exposure time obtained and arequired image brightness, the image capture device is configured, andthen an image is captured by the image capture device. Once the imagehas been captured, due to the possibility of under-exposure from havinglimited the exposure time, the image is processed as required toincrease its brightness intensity.

The motion blur limiting method is particularly suited for use inconjunction with algorithms that automatically select exposure timesbased on only intensity requirements, in order to limit exposure timewhere the automatically-selected intensity-based exposure time wouldresult in an unacceptable blur extent.

The motion blur limiting method and apparatus may be embodied in animage capture device such as a digital camera. FIG. 1 is a block diagramof a motion blur limiting apparatus 50, and FIG. 2 is a block diagram ofan image capture device 60 incorporating the motion blur limitingapparatus 50 of FIG. 1.

Motion blur limiting apparatus 50 comprises gyros 52 a, 52 b, 52 c formeasuring a movement rate of the image capture device 60 in respectiveones of three (3) dimensions (X, Y, Z), analog-to-digital converters 53a, 53 b, 53 c for converting the analog signals from the gyros 52 a to52 c to digital signals, a processor 54, and a processor-readablestorage device 56 for storing a software application embodied asprocessor-readable code which may include program modules such asroutines, programs, object components, data structures, lookup modulesetc. Based on data received from the gyros 52 a to 52 c, processor 54retrieves, calculates or otherwise obtains from lookup tables and/orprocessor-readable code on storage device 56 a maximum exposure time forthe image capture device 60 for limiting motion blur, as will bedescribed.

The image capture device 60 comprises an image sensor 62 for capturingan image, a brightness detector 63 for detecting brightness of asubject, a processor 64 for configuring and controlling image sensorsensitivity, focal length, aperture, exposure time etc. of image capturedevice 60, a storage device 66 for storing processor-readable code suchas routines, programs, object components, data structures etc. used byprocessor 64, and a user interface 68 for receiving user input settingsused by processor 64 and storable by data storage device 66. The imagesensor includes a charge-coupled device (“CCD”); that is, alight-sensitive integrated circuit that registers the data for an imagein such a way that each pixel (picture element) in the image isconverted into an electrical charge the intensity of which is related toa color in the color spectrum. Based on the maximum exposure timeobtained by the motion blur limiting apparatus 50, image capture device60 configures the image sensor 62 in accordance with detected subjectbrightness, captures an image using the image sensor 62, and brightensthe captured image if it has been underexposed.

It will be understood that, based on design considerations, componentsof motion blur limiting apparatus 50 and image capture device 60 may becombined. For example, processors 54 and 64 may be the same processor.Similarly, storage device 56 and storage device 66 may be the samestorage device.

Turning now to FIG. 3, a flowchart showing the general steps performedto limit blur in a captured image is shown and generally identified byreference numeral 90. During blur extent limiting, a relationship isinitially determined between movement of the image capture device, timeand blur extent (step 100). A rate of movement of the image capturedevice is then obtained (step 200) and is used in conjunction with thedetermined relationship and a blur extent limit as a basis for obtaininga maximum exposure time for the image capture device (step 300). On thebasis of the maximum exposure time obtained and a required imagebrightness, the image capture device is configured (step 400), and theimage is captured by the image capture device (step 500). Once the imagehas been captured, it is processed as required to increase its intensity(i.e., brightness) (step 600).

FIG. 4 is a flowchart showing the steps performed in order to determinethe relationship between movement of the image capture device, time andblur extent at step 100 of FIG. 3. First, a relationship between worldobject coordinates and image pixel coordinates in the image capturedevice is defined (step 110). This relationship is defined using asimple lens model.

FIG. 5 is a schematic diagram showing the defined image capture devicemodel that is based on the simple lens model. As will be understood, thefocal plane is parallel to the image plane. Based on an assumption thatthe lens employed is relatively thin and that its optical axis isperpendicular to the image plane, the lens operates according to thefollowing lens law:

$\begin{matrix}{{\frac{1}{u} + \frac{1}{v}} = \frac{1}{f}} & (1)\end{matrix}$where:

u is the distance of a world object point from the lens plane;

v is the distance of the focused image from the lens plane; and

f is the focal length of the lens.

The following discussion of the relationship between world objectcoordinates and image pixel coordinates employs the following terms:

F—Image capture device focal length;

P_(w)=(x_(w),y_(w),z_(w))—Location of a point in world objectcoordinates;

P_(c)=(x_(c),y_(c),z_(c))—Location of a point in image capture devicecoordinates;

P_(im)=(x_(im)y_(im))—Location of a point in image plane coordinates;

P_(focused)=(x_(focused),y_(focused))—Location of a point in focal planecoordinates;

P=(x, y)—Location of a point in image pixel coordinates;

S_(x)—Horizontal lens scaling factor;

S_(y)—Vertical lens scaling factor;

x₀—Horizontal principal point;

y₀—Vertical principal point;

λ—Model scaling factor;

α—Angle of image capture device rotation around X axis;

β—Angle of image capture device rotation around Y axis; and

θ—Angle of image capture device rotation around Z axis.

The relationship between world object coordinates and image pixelcoordinates comprises four components. The first component of therelationship is a rotation and translation between the image capturedevice and world object coordinates:P _(c) =R·P _(w) +T   (2)where:

R is a 3×3 rotation matrix defining rotations about X, Y and Z; and

T is a 3D translations vector defining translation between origins.

The second component of the relationship is a perspective projectionfrom the world object coordinates to the image plane. This perspectiveprojection is based on the simplified pinhole camera projection model:

$\begin{matrix}{\frac{x_{im}}{f} = {{\frac{x_{c}}{z_{c}}\mspace{14mu}{and}\mspace{14mu}\frac{y_{im}}{f}} = \frac{y_{c}}{z_{c}}}} & (3)\end{matrix}$

The third component, according to the simple lens model shown in FIG. 7,relates the focal plane of the image capture device and the image planewith the scaling factor λ that is a function of (u, v, f) as follows:y _(focused) =y _(im)*λ and x _(focused) =x _(im)*λ  (4)where:

(x_(im), y_(im)) is the projected point in 2D image plane coordinates;

(x_(focused), y_(focused)) is the corresponding point in 2D focal planecoordinates; and

λ is proportional to v/f and close to 1.

The fourth component of the relationship is a transformation between thefocal plane and image pixel coordinates:(y−y ₀)=y _(focused) /S _(y) and (x−x ₀)=x _(focused) /S _(x)   (5)

The relationship between the world object coordinates and the imagepixel coordinates is therefore a combination of Equations (2), (3), (4)and (5), as follows:

The coordinate projection CP can therefore be expressed as:

$\begin{matrix}{{CP} = {\begin{bmatrix}{\lambda\;\frac{F}{S_{x}}} & 0 & 0 \\0 & {\lambda\;\frac{F}{S_{y}}} & 0 \\0 & 0 & 1\end{bmatrix} = \begin{bmatrix}{{Fc}\; 1} & 0 & 0 \\0 & {{Fc}\; 2} & 0 \\0 & 0 & 1\end{bmatrix}}} & (6) \\{{and}\mspace{14mu}{as}\text{:}} & \; \\{\begin{bmatrix}{x - x_{0}} \\{y - y_{0}} \\1\end{bmatrix} = {{{CP} \cdot \begin{bmatrix}x_{c} \\y_{c} \\z_{c}\end{bmatrix}} = {{CP} \cdot \left( {{R \cdot \begin{bmatrix}x_{w} \\y_{w} \\z_{w}\end{bmatrix}} + T} \right)}}} & (7)\end{matrix}$

The horizontal and vertical scaling factors S_(x) and S_(y), arecalculated from the resolution of the image sensor 62 on the focalplane. The scaling factor λ is calibrated.

Based on the relationship between world object coordinates and imagepixel coordinates defined in step 110, a model relating movement rate ofthe image capture device, time and extent of pixel movement is derived(step 112). According to this embodiment, the particular movement rateemployed is angular velocity, as will be described.

The movement rate model described herein is, for simplicity and ease ofunderstanding, based on an assumption that image capture device motionduring the time the shutter is open is purely rotational. Under thisassumption, the translation vector T is a zero vector:

$\begin{matrix}{T = \begin{bmatrix}0 \\0 \\0\end{bmatrix}} & (8)\end{matrix}$

As a result, the image capture device model of Equation (7) can besimplified as follows:

$\begin{matrix}{\begin{bmatrix}{x - x_{0}} \\{y - y_{0}} \\1\end{bmatrix} = {{{CP} \cdot \begin{bmatrix}x_{c} \\y_{c} \\z_{c}\end{bmatrix}} = {{CP} \cdot R \cdot \begin{bmatrix}x_{w} \\y_{w} \\z_{w}\end{bmatrix}}}} & (9)\end{matrix}$

The rotation matrix R in Equation (9) is calculated as follows:R=R _(x) ·R _(y) ·R _(z)   (10)

The matrix representing rotation about the X-axis is:

$\begin{matrix}{R_{x} = \begin{bmatrix}1 & 0 & 0 \\0 & {\cos\;\alpha} & {\sin\;\alpha} \\0 & {{- \sin}\;\alpha} & {\cos\;\alpha}\end{bmatrix}} & (11)\end{matrix}$

The matrix representing rotation about the Y-axis is:

$\begin{matrix}{R_{y} = \begin{bmatrix}{\cos\;\beta} & 0 & {\sin\;\beta} \\0 & 1 & 0 \\{{- \sin}\;\beta} & 0 & {\cos\;\beta}\end{bmatrix}} & (12)\end{matrix}$

The matrix representing rotation about the Z-axis is:

$\begin{matrix}{R_{z} = \begin{bmatrix}{\cos\;\gamma} & {\sin\;\gamma} & 0 \\{{- \sin}\;\gamma} & {\cos\;\gamma} & 0 \\0 & 0 & 1\end{bmatrix}} & (13)\end{matrix}$

As defined above, α, β, and γ are the angles rotated about the X-, Y-and Z-axes, respectively, and are related to angular velocity andexposure time as follows:

$\begin{matrix}{{\alpha = {{\int_{0}^{E}{G_{x}\ {\mathbb{d}t}}} = {\frac{E}{N}{\sum\limits_{i = 1}^{N}{G_{x}(i)}}}}}{\beta = {{\int_{0}^{E}{G_{y}\ {\mathbb{d}t}}} = {\frac{E}{N}{\sum\limits_{i = 1}^{N}{G_{y}(i)}}}}}{\gamma = {{\int_{0}^{E}{G_{z}\ {\mathbb{d}t}}} = {\frac{E}{N}{\sum\limits_{i = 1}^{N}{G_{z}(i)}}}}}} & (14)\end{matrix}$where:

E is exposure period;

G_(x), G_(y) and G_(z) are angular velocities obtained from the three(3) gyros 52 a to 52 c as will be described below; and

N is the number of gyro samples obtained during exposure period E.

Prior to rotation, image capture device coordinates are aligned with theworld object coordinates. Therefore, the rotation matrix prior torotation is:

$\begin{matrix}{R = {I = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}}} & (15)\end{matrix}$

With no rotation, the image pixel coordinates corresponding to the worldobject point P_(w)=(x_(w),y_(w),z_(w)) can be represented by:

$\begin{matrix}{\begin{bmatrix}{x_{1} - x_{0}} \\{y_{1} - y_{0}} \\1\end{bmatrix} = {{{CP} \cdot R \cdot \begin{bmatrix}x_{w} \\y_{w} \\z_{w}\end{bmatrix}} = {{CP} \cdot \begin{bmatrix}x_{w} \\y_{w} \\z_{w}\end{bmatrix}}}} & (16)\end{matrix}$where:

(x₁,y₁) are image pixel coordinates corresponding to point P_(w) inworld object coordinates.

After rotation, which as described above is assumed to be pure rotationabout X, Y and Z axis while the shutter of the image capture device isopen, the rotated angles are α, β, and γ respectively. The rotationmatrix R is represented by Equation (10). Furthermore, the image pixelcoordinates corresponding to the world object pointP_(w)=(x_(w),y_(w),z_(w)) prior to image capture device rotation arerepresented by Equation (16). From Equation (16), pointsP_(w)=(x_(w),y_(w),z_(w)) in the world object coordinates can berepresented as follows:

$\begin{matrix}{\begin{bmatrix}x_{w} \\y_{w} \\z_{w}\end{bmatrix} = {{CP}^{- 1} \cdot \begin{bmatrix}{x_{1} - x_{0}} \\{y_{1} - y_{0}} \\1\end{bmatrix}}} & (17)\end{matrix}$

For simplicity and clarity of description, it is assumed that thesubject to be captured by the image capture device does not move duringexposure. By substituting Equation (17) into Equation (9), arelationship for each pixel before and after rotation, in imagecoordinates, is obtained:

$\begin{matrix}{\begin{bmatrix}{x_{2} - x_{0}} \\{y_{2} - y_{0}} \\1\end{bmatrix} = {{{CP} \cdot R \cdot \begin{bmatrix}x_{w} \\y_{w} \\z_{w}\end{bmatrix}} = {{CP} \cdot R \cdot {CP}^{- 1} \cdot \begin{bmatrix}{x_{1} - x_{0}} \\{y_{1} - y_{0}} \\1\end{bmatrix}}}} & (18)\end{matrix}$where:

(x₂,y₂) is the image pixel coordinates of image corresponding to worldobject point P_(w) after rotation of the image capture device; and

(x₁,y₁) is the image pixel coordinates corresponding to world objectpoint P_(w) prior to rotation of the image capture device.

With the model relating movement rate of the image capture device, timeand extent of pixel movement having been determined at step 112, anumber of combinations of focal length, movement rate (angular velocity)and time are inserted into the model to pre-define pixel movement asfields of motion vectors in each of the X-, Y- and Z-axis directions(step 114). A motion vector for each pixel in an image can berepresented by:

$\begin{matrix}{\begin{bmatrix}{x_{2} - x_{1}} \\{y_{2} - y_{1}} \\0\end{bmatrix} = {\begin{bmatrix}{x_{2} - x_{0}} \\{y_{2} - y_{0}} \\1\end{bmatrix} - \begin{bmatrix}{x_{1} - x_{0}} \\{y_{1} - y_{0}} \\1\end{bmatrix}}} & (19)\end{matrix}$

FIGS. 6, 7 and 8 are diagrams showing fields of motion vectorsrepresenting rotation about the X-, Y- and Z-axes, respectively, forparticular combinations of focal length, angular velocity, and time. InFIG. 6, under pure rotation about the X-axis, for each point P=(x,y) inthe image plane, the dominant motion vector is along the Y coordinatewith a very slight shift in the X-coordinate. In FIG. 7, pure rotationabout the Y-axis is shown, and for each point P=(x,y) in the imageplane, the dominant motion vector is along the X coordinate with a veryslight shift in the Y coordinate. Pure rotation around the Z-axis isshown in FIG. 8.

A motion vector field is determined for each of the predeterminedcombinations. The combinations may be chosen during design of the imagecapture device 60 or motion blur limiting apparatus 50 based on how manyfocal lengths in a range are available using the image capture device,how many exposure times in a range may be set in the image capturedevice, and how many angular velocities in a range may be measured inthe image capture device. It will be understood that the number ofcombinations may be limited by the physical capacity of the imagecapture device 60 or motion blur limiting apparatus 50 to store themultiple relationships, or other such design factors.

Once a field of motion vectors has been pre-defined for a particularrotation direction, a weighted average motion vector for each field isobtained (step 116):

$\begin{matrix}\begin{matrix}{{WeightedAvgMotionVector} = \begin{bmatrix}{\Delta\;\overset{\_}{x}} \\{\Delta\;\overset{\_}{y}}\end{bmatrix}} \\{= {\frac{1}{W*H}{\sum\limits_{x = 0}^{W - 1}{\sum\limits_{y = 0}^{H - 1}{M\left( {x,y} \right)}}}}} \\{{Weight}\;\left( {x,y} \right)}\end{matrix} & (20)\end{matrix}$where:

M(x,y) is the motion vector; and

Weight(x,y) is a weight for the pixel at coordinates (x,y).

The pixel weighting is applied in such a manner as to afford more weightto more centrally-located pixels in the field. It will be understoodthat pixel weighting may be adjusted experimentally based on designfactors.

The extent and angle of the weighted average motion vector and thereforethe motion of the pixels are calculated as follows:

$\begin{matrix}{{MotionExtent} = \sqrt{\left( {{\Delta\;{\overset{\_}{x}}^{2}} + {\Delta\;{\overset{\_}{y}}^{2}}} \right)}} & (21) \\{{MotionAngle} = {\arctan\left( \frac{\Delta\;\overset{\_}{y}}{\Delta\;\overset{\_}{x}} \right)}} & (22)\end{matrix}$

The motion extent is the blur extent resulting from a particularcombination of angular velocity and time (step 118). In order tosimplify calculation of motion extents and angle, the weighted averagemotion vectors of nine (9) pixels are used to approximate the overallmotion vector. Table 1 below shows the pixel locations and correspondingweights of the selected nine (9) pixels.

TABLE 1 left-top: middle-top: right-top: (0, 0) (W/2, 0) (W-1, 0)weight: 1/16 weight: 2/16 weight: 1/16 left-centre: middle-centre:right-centre: (0, H/2) (W/2, H/2) (W-1, H/2) weight: 2/16 weight: 4/16weight: 2/16 left-bottom: middle-bottom: right-bottom: (0, H) (W/2, H)(W-1, H-1) weight: 1/16 weight: 1/16 weight: 1/16

As described above, in order to reduce the amount of calculation by theimage capture device during image capture, as well as to lessen the needfor a powerful on-board processor, lookup tables for determining amaximum exposure time are pre-calculated. That is, for a given exposuretime and angular velocity, motion (i.e. blur) extents can bepre-calculated using Equation (21). An exposure limit lookup table isdetermined for each combination of focal length-and angular velocity,which stores the maximum exposure time that would result in an extent ofmotion blur that is acceptable.

Table 2 below is a one-dimensional lookup table containingpre-calculations of blur extent for image capture device rotation purelyabout the X-axis.

TABLE 2 ang. Vel. Exp. time 20 35 50 65 80 0.10 23.5 41.2 58.9 76.8 94.80.066 15.5 27.2 38.8 50.5 62.3 0.033 7.8 13.7 19.6 25.5 31.4 0.016 3.86.6 9.4 12.2 15.0 0.008 1.9 3.3 4.7 6.1 7.5 0.006 1.4 2.5 3.5 4.6 5.6

A number of acceptable blur extents may be selectable by a user. Lookuptables may be created using combinations of exposure time and angularvelocity that result in the acceptable blur extents (step 120). For anacceptable blur extent of ten (10) pixels, an exposure limit lookuptable can be obtained from Table 2, for each angular velocity,identifying the maximum exposure times resulting in motion extents lessthan or equal to the acceptable level (step 122). Table 3 below is suchan exposure limit lookup table for an acceptable blur extent of ten (10)pixels.

TABLE 3 Angular Velocity 20 35 50 65 80 E_(max) for 10 pixels 0.0330.016 0.016 0.008 0.008

Table 4 below is an exposure limit lookup table for an acceptable blurextent of twenty-eight (28) pixels.

TABLE 4 Angular Velocity 20 35 50 65 80 E_(max) for 28 pixels 0.10 0.0660.033 0.033 0.016

Preferably, a three-dimensional exposure limit lookup table thatsupports image capture device rotation about X, Y and Z axis isemployed. An example exposure limit lookup table entry for a particularextent of blur is shown in Table 5 below.

TABLE 5 Angular Velocity Index Exposure Limit XX YY ZZ E_(max)

A linear index may be employed for the exposure limit lookup table. Thatis, the spacing between velocities in the table may be linear. However,depending upon design considerations, one may wish to employ alogarithmic distribution of G_(x), G_(y), and G_(z) in order to generatethe lookup table. Such a logarithmic index would result in a higherproportion of exposure limits at very low angular velocities, in orderto more accurately track an exposure limit for respective ones of theangular velocities at which small jitter tends to occur as a result ofhand movement.

For a linear index, the angular velocity or gyro index values XX, YY andZZ are calculated as follows:XX=|G _(x)|*(n−1)/G _(max)YY=|G _(y)|*(n−1)/G _(max)ZZ=|G _(z)|*(n−1)/G _(max)   (23)where:

G_(max) is the maximum angular velocity of gyro output; and

n is the number of possible indices for each gyro in the exposure limitlookup table.

As can be seen, the index is scaled between 0 and n−1. The value of ncan be varied based on the level of precision needed for the exposurelimit lookup table. During testing, n=32 was used.

For a logarithmic index, the gyro index values XX, YY, ZZ are calculatedas follows:XX=log₁₀(|G _(x)|*10)*(n−1)/log₁₀(G _(max)*10)YY=log₁₀(|G _(y)|*10)*(n−1)/log₁₀(G _(max)*10)  (24)ZZ=log₁₀(|G _(z)|*10)*(n−1)/log₁₀(G _(max)*10)

Once the relationship between movement of the image capture device, timeand blur extent has been determined and stored in lookup tables asdescribed above, the image capture device 60 is ready for image capture.

FIG. 9 is a flowchart showing steps for obtaining a rate of movement ofthe image capture device 60 from the gyros 52 a to 52 c for use indetermining a corresponding maximum exposure time from the lookuptables, and ultimately for use in controlling the exposure time of theimage capture device 60. First, an analog voltage output from each ofthe X, Y and Z gyros 52 a to 52 c is obtained (step 210). The voltageamounts are then sampled and amplified (step 212), and outputted fromanalog-to-digital converters as digital values (step 214). In order toyield true angular velocities, the digital values are corrected forstatic (i.e., zero-velocity) gyro output amounts (step 216) and furthercorrected for sensor gain, sensitivity, and voltage step size of theanalog-to-digital converters (step 218). The following formulas are usedto obtain the output angular velocities from the digital gyro outputs:G _(x)=(G _(—) ADC _(x) −G_Offset_(x))*G _(—) AdcStepSize/G _(—)AdcConvertFactor)G _(y)=(G _(—) ADC _(y) −G_Offset_(y))*G _(—) AdcStepSize/G _(—)AdcConvertFactor)G _(z)=(G _(—) ADC _(z) −G_Offset_(z))*G _(—) AdcStepSize/G _(—)AdcConvertFactor)  (25)where:

G_(x), G_(y) and G_(z) are angular velocities;

G_ADC_(x), G_ADC_(y) and G_ADC_(z) are the digital gyro outputs;

G_Offset_(x), G_Offset_(y) and G_Offset_(z) are zero offsets;

G_AdcStepSize is the gyro analog-to-digital voltage step size; and

G_AdcConvertFactor is the gyro analog-to-digital conversion factor.

G_AdcStepSize is calculated as follows:G _(—) AdcStepSize=MaximumGyroVoltage/AdcStepNumber   (26)

G_AdcConvertFactor is calculated as follows:G _(—) AdcConvertFactor=GyroSensitivity*AdcGain   (27)

A testing platform employing a PENGUIN board manufactured by Pixera™ ofLos Gatos, Calif., U.S.A. with 10-bit ADC values and a 3.3 volt gyrooutput range was used. The gyro sensitivity was 0.67 mv/dps and theAdcGain was 100. As a result, the testing platform had a gyroanalog-to-digital voltage step size G_AdcStepSize=3300/2¹⁰=3.22 and agyro analog-to-digital conversion factor G_AdcConvertFactor=0.67*100=67.

With the angular velocities representing movement rate of the imagecapture device in the X-, Y- and Z-dimensions having been determined, amaximum exposure time E_(max) that is based on the lookup tables, theangular velocities and a blur extent limit is obtained. FIG. 10 is aflowchart showing steps for obtaining a maximum exposure time E_(max)based on the data in the lookup tables, the angular velocity of theimage capture device and the blur extent limit. First, a predeterminedblur extent limit is retrieved (step 310). This blur extent limit may beadjustable by a user to one of several available levels within a range.The focal length of the image capture device is then determined (step312), and based on the focal length and blur extent limit, a lookuptable (such as that shown in Table 5) is automatically chosen (step314). Based on the detected X-, Y- and Z-angular velocities, a maximumexposure time E_(max) is selected from the chosen lookup table (step316).

With the maximum exposure time E_(max) having been selected from thelookup tables based upon the angular velocities and the predeterminedblur extent limit, the image capture device 60 is configured to capturean image based upon the maximum exposure time E_(max) and required imageintensity. FIG. 11 is a flowchart showing steps for configuring theimage capture device based on the maximum exposure time E_(max) andrequired image intensity. According to this embodiment, the configuringsteps make use of a standard intensity-based auto-exposure function ofthe image capture device 60.

First, the exposure time E_(true) of the image capture device is set tothe lower of E_(max) and the standard (intensity-based) automaticexposure time setting E_(auto) of the image capture device (step 410):E _(true)=min(E _(auto) ,E _(max))  (28)

If the automatic exposure time E_(auto) is less than or equal to themaximum exposure time E_(max) (step 412), then the image capture deviceretains the automatic brightness-based settings (step 414). In thiscase, no adjustments to the exposure time E_(true) need to be madebecause any blur occurring during image capture will not exceed the blurextent limit corresponding to E_(max). If, however, the automaticexposure time E_(auto) is greater than E_(max), then any resultant blurextent will be greater than the blur extent limit setting. In this case,the shutter speed of the image capture device is set to E_(max) (step416). Based on the adjusted shutter speed, the aperture size and ISOsettings of the image capture device are automatically adjusted (step418) in order to preserve intensity of a captured image.

As will be understood, according to the Additive Photographic ExposureSystem, exposure is balanced as follows:Av+Tv=Sv+Bv   (29)where:

Av=Aperture value;

Tv=Shutter speed or time value;

Sv=Sensitivity value or film speed value; and

Bv=Brightness value of subject.

These values are calculated as follows:Sv=log₂(ISO*N)  (30)Bv=log₂(B/NK)  (31)Av=log₂(F ²)  (32)Tv=log₂(1/T)  (33)

Advantageously, the standard auto-exposure function of many imagecapture devices such as digital cameras and the like calculate theexposure time automatically time according to Equation (29), resultingin:Av _(auto) +Tv _(auto) =Sv _(auto) +Bv   (34)

When exposure time E_(true) is reduced, shutter speed is increased:Tv _(true)=log₂(1/E _(true))>Tv _(auto)=log₂(1/E _(auto))  (35)

It is reasonable to assume that for most cases, the focal length andbrightness value of the subject to be captured by the image capturedevice will remain constant. In order to ensure sufficient exposureaccording to Equation (29), Av may be adjusted by changing the aperturesize, Sv may be adjusted by changing the ISO setting, or both Av and Svmay be adjusted. A combination of the two adjustments supports a largerange of exposure times.

Adjustments to aperture size Av only will now be described. It will beunderstood that the f-stop number is the relative aperture of the lens,indicating the amount of light that the lens lets into the image capturedevice:

$\begin{matrix}{{f\_ stop} = \frac{FocalLength}{ApertureSize}} & (36)\end{matrix}$

As long as the focal length is not changed, an increase in aperture sizewill decrease the f-stop. Furthermore, according to Equation (32),decreasing the f-stop decreases the aperture value, Av.

An exposure value (EV) is defined as:EV=Av+Tv=Sv+Bv   (37)

If only the aperture size is adjusted, exposure value EV will not bechanged because Sv and Bv are not changed. For proper exposure, E_(true)and Av_(true) are set to achieve the same exposure value as that ofE_(auto) and Av_(auto):Av _(auto) +TV _(auto) =Av _(true) +TV _(true)   (38)

Combining Equations (32) and (33), the following is obtained:log₂(1/E _(auto))+log₂(F _(auto) ²)=log₂(1/E _(true))+log₂(F _(true)²)  (39)

The f-stop value F_(true) is obtained according to:

$\begin{matrix}{\frac{F_{true}}{F_{auto}} = \sqrt{\frac{E_{true}}{E_{auto}}}} & (40)\end{matrix}$

The f-stop value for a specified E_(true) can be calculated usingEquation (40), or an exposure control table such as that shown in Table6 below can yield the closest pair of aperture value and shutter speed.

For example, when the automatic exposure time is 1/125 and the f-stop is16, thenEV_(auto)=log₂(1/E_(auto))+log₂(F²)=log₂(125)+log₂(162)=6.97+8=14.98, orapproximately 15.

If E_(true) is 1/500, the f-stop value can be calculated by Equation(40), F_(true)=16/2=8. The exposure value, EV_(true) is about 15.

TABLE 6 f-stop f/5.6 f/8 f/11 f/16 shutter _(TV)\^(AV) 5 6 7 8 1/125 712 13 14 15 1/250 8 13 14 15 16 1/500 9 14 15 16 17 1/1000 10  15 16 1718

Since the aperture size is only selectable from discrete values, theexposure value table of the shutter speed and aperture size is used toselect a proper (F′_(true), E′_(true)) pair which would yield theclosest exposure value and aperture value with shutter speed within theexposure limit. This can be implemented in the form of a function thatreferences the exposure value table:(F _(true) ,E _(true))=RoundUp(F _(true) ,E _(true))  (41)

If E_(true) is 1/800, and exposure value is 15, the f-stop value can becalculated using Equation (40), F_(true)=6.32. According to Table 6,there are four combinations that will yield an exposure value of 15:1/125 sec@f/16, 1/250 sec@ f11, 1/500@ f8 and 1/1000 sec@ f5.6. Amongthe options yielding an exposure value of 15, the (F′_(true),E′_(true))pair having a shutter speed within the limit is 1/1000 sec @ f5.6. Theexposure time E_(true)= 1/800 will be adjusted to E′_(true)= 1/1000, andF_(true)=F′_(true)= f5.6.

If F_(true) is smaller than a minimum aperture value, F_(min), there isno (F′_(true), E′_(true)) combination in the exposure control table thatwould achieve the proper exposure value. In this case, the aperture sizeis set to the biggest aperture size available and E_(true) is used tocapture the image:F _(true)=max(F _(true) ,F _(min))  (42)

Adjustments to ISO setting Sv only will now be described. Equation (29)may be written as follows:Av−Bv=Sv−Tv   (43)

If only Sv is adjusted, and aperture value Av is not changed, then Av−Bvwill not change. According to Equation (43), the condition for properexposure can be expressed as:Sv _(auto) −Tv _(auto) =Sv _(true) −Tv _(true)   (44)

Based on Equations (30) and (33), Equation (44) can be written asfollows:log₂(ISO _(auto) *N)−log₂(1/E _(auto))=log₂(ISO _(true)*N)−log₂(1/E_(true))  (45)

ISO_(true) can be obtained as follows:

$\begin{matrix}{\frac{{ISO}_{true}}{{ISO}_{auto}} = \frac{E_{auto}}{E_{true}}} & (46)\end{matrix}$

For example, in the event that the automatic exposure time, E_(auto), is1/125, the ISO setting ISO_(auto) is 100 and E_(true)= 1/500, thenISO_(true) will be 400.

When increasing the sensitivity, the output of the sensor is amplified,so less light is required. However, it should be understood thatamplification of the sensor undesirably amplifies noise also.Furthermore, when exposure time is reduced to a very short period inorder to prevent motion blur in the cases of large amounts of movement,or when very low light conditions are encountered, an under-exposureimage may be obtained even with a maximum aperture size. As such,adjustment of both Av and Sv may achieve better quality of capturedimage than adjustment of only one. As a basis for adjusting both Av andSv, Equation (29) may be presented as follows:Av+Tv−Sv=Bv   (47)

On the assumption that brightness Bv of the subject does not change, thefollowing is obtained:Av _(auto) +Tv _(auto) −Sv _(auto) =Av _(true) +Tv _(true) −SV _(true)  (48)

Based on Equations (30), (32), (33) and (48), the adjustment of both theaperture and ISO settings can be related as follows:

$\begin{matrix}{{\frac{F_{auto}^{2}}{F_{true}^{2}}*\frac{{ISO}_{true}}{{ISO}_{auto}}} = \frac{E_{auto}}{E_{true}}} & (49)\end{matrix}$

In order to minimize the noise captured due to high ISO settings, theaperture size is adjusted first according to Equations (41) and (42),and then the ISO settings are adjusted according to Equation (49) asfollows:

$\begin{matrix}{{ISO}_{true} = {\frac{E_{auto}}{E_{true}}\frac{F_{true}^{2}}{F_{auto}^{2}}*{ISO}_{auto}}} & (50)\end{matrix}$

With the image capture device having been configured based on themaximum exposure time E_(max) and required image intensity, the imagecapture device 60 is ready for image capture (step 500). Once an imageis captured, it is processed as described below in order to brighten theimage if it is underexposed by increasing the intensity of pixels in theimage.

It is generally understood that electrons are freed when photons strikethe silicon surface of a CCD of the image sensor 62. The electrons aretransferred to a capacitor, and the voltage induced by the chargedcapacitor is measured, amplified and finally digitalized using ananalog-to-digital converter.

The arrival of photons is modeled as a Poisson distribution, having thegeneral property that the possibility of event occurrence isproportional to the duration of observation. As such, the arrival ofphotons is modeled as:U=λ*T   (51)where:

λ=E(X) is the mean value of the Poisson distribution; and

T is the exposure time.

The gain of the image capture device is defined in decibels (dB), whichis one-tenth of the common logarithm of the ratio of relative powers:

$\begin{matrix}{{dB} = {10\log_{10}\frac{P_{2}}{P_{1}}}} & (52)\end{matrix}$where:

P₁ and P₂ are the actual measures of power.

Power ratios may be expressed in terms of voltage and impedance, U andR, since:P=U ² /R   (53)

Combining Equations (52) and (53), the dB gain can be expressed as:

$\begin{matrix}{{dB} = {{10\mspace{11mu}\log_{10}\frac{P_{2}}{P_{1}}} = {{10\mspace{11mu}\log_{10}\frac{U_{2}^{2}/R}{U_{1}^{2}/R}} = {20\mspace{11mu}\log_{10}\frac{U_{2}}{U_{1}}}}}} & (54)\end{matrix}$

For a digital camera in particular, the intensity of pixel value isproportional to output U₂ of the amplifier. The input signal U₁ isproportional to the arrival of photons. The gain of the digital cameracan be represented in terms of pixel intensity value and the arrival ofphotons:

$\begin{matrix}\begin{matrix}{{Gain} = {20\mspace{11mu}\log_{10}\frac{U_{2}}{U_{1}}}} \\{= {20\mspace{11mu}\log_{10}\frac{{Intensity}*k_{1}}{\lambda*T*k_{2}}}} \\{= {20\mspace{11mu}\log_{10}\frac{Intensity}{\lambda*T}k}}\end{matrix} & (55)\end{matrix}$where:

k=k₁/k₂ is a constant; and

T is exposure time.

It is assumed that E_(auto) and G_(auto) can achieve a proper exposure.If it is desired to achieve the same exposure using E_(true) andG_(true), the following relationships based on Equation (55) areobtained:

$\begin{matrix}{{\frac{{Intensity}_{true}}{\lambda*E_{true}}k} = 10^{G_{true}/20}} & (56) \\{{\frac{{Intensity}_{auto}}{\lambda*E_{auto}}k} = 10^{G_{auto}/20}} & (57)\end{matrix}$

Combining Equations (56) and (57), the following is obtained:

$\begin{matrix}{{\frac{{Intensity}_{true}}{{Intensity}_{auto}}*\frac{E_{auto}}{E_{true}}} = \frac{10^{G_{true}/20}}{10^{G_{auto}/20}}} & (58)\end{matrix}$

As will be understood, the relationship between an ISO setting and gainvalue is defined as:IsoValue=BaseIsoValue*10^((GainValue−BaseGain)*/20)   (59)

Using Equation (59), the relationship between G_(auto), G_(true) andISO_(true), ISO_(auto) can be expressed as follows:

$\begin{matrix}{\frac{{ISO}_{true}}{{ISO}_{auto}} = {\frac{{BaseIsoValue}*10^{{({G_{true} - {BaseGain}})}*{/20}}}{{BaseIsoValue}*10^{{({G_{auto} - {BaseGain}})}*{/20}}} = \frac{10^{G_{true}/20}}{10^{G_{auto}/20}}}} & (60)\end{matrix}$

By combining Equation (60) with Equation (59) a relationship betweenintensity value, exposure time and ISO settings can be obtained:

$\begin{matrix}{\frac{{Intensity}_{true}}{{Intensity}_{auto}} = {\frac{{ISO}_{true}}{{ISO}_{auto}}*\frac{E_{true}}{E_{auto}}}} & (61)\end{matrix}$

The gain compensation for proper exposure given a reduced exposure timemay be expressed as follows:

$\begin{matrix}{\frac{{ISO}_{true}}{{ISO}_{auto}} = \frac{E_{auto}}{E_{true}}} & (62)\end{matrix}$

Under proper conditions of exposure using gain compensation, theintensity value of an image captured using E_(auto) and G_(auto) will bethe same as the intensity value of an image captured using E_(true) andG_(true) by substituting Equation (62) into Equation (61). That is:

$\begin{matrix}{\frac{{Intensity}_{true}}{{Intensity}_{auto}} = 1} & (63)\end{matrix}$

However, if there is either no gain compensation or insufficient gaincompensation, the intensity value of the captured image taken usingE_(true) and G_(true) will be darker than the intensity value of acaptured image taken using E_(auto) and G_(auto):

$\begin{matrix}{\frac{{Intensity}_{true}}{{Intensity}_{auto}} = {{\frac{{ISO}_{true}}{{ISO}_{auto}}*\frac{E_{true}}{E_{auto}}} < 1}} & (64)\end{matrix}$

In order to brighten an underexposed captured image using E_(true) andG_(true), the intensity must be corrected by a “brighten” factor:

$\begin{matrix}{{BrightenFactor} = {\frac{{ISO}_{auto}}{{ISO}_{true}}*\frac{E_{auto}}{E_{true}}}} & (65)\end{matrix}$

BrightenFactor is 1 if the proper exposure has been achieved by gaincompensation:Intensity_(brightened)=Intensity_(true)*BrightenFactor   (66)

It will be understood that Equation (66) is valid only in the case ofideal CCD sensors. If perfect CCDs were employed, there would be adirect relationship between the amount of light going into the lens andthe brightness of the resultant pixels. That is, if brightness goinginto the lens were increased by two, the resultant pixels would be twiceas bright. It will be understood, however, that real CCD sensors undergosaturation at high intensity.

A simple model of saturation during brightening is obtained by measuringa characteristic curve of CCD exposure, and then simulating an exposureprocess. Using 128 as the ‘middle’ exposure intensity, a saturationmodel is based on the following function:

$\begin{matrix}{{Intensity}_{saturated} = \left\{ \begin{matrix}\frac{255}{1 + {128/{Intensity}_{brighten}}} & {{Intensity}_{brighten}>=128} \\{Intensity}_{brighten} & {{Intensity}_{brighten} < 128}\end{matrix} \right.} & (67)\end{matrix}$

FIG. 12 shows both a linear intensity saturation curve and a logarithmicintensity saturation curve. On the basis of the saturation model, thecaptured image may be brightened. Referring to the flowchart of FIG. 13,the captured image is first converted to YIQ color space and theintensity channel Y obtained (step 610). The E_(true), E_(auto),ISO_(true) and ISO_(auto) settings that were used during image captureare retrieved (step 612) and the brighten factor BrightenFactor iscalculated using Equation (65) (step 614). If the BrightenFactor is 1 orless than 1 (step 616), then application of BrightenFactor would eitherdarken the captured image or have no effect, so simply the image ascaptured is output as the post-processed image (step 618). If, however,BrightenFactor is greater than 1, it is applied to the intensity channelY of the captured image according to Equation (66) (step 620). Accordingto Equation (67), the brightened pixel intensities are then saturated(step 622).

After the brightened pixel intensities have been saturated, the modifiedintensity channel Y replaces the one extracted from the YIQ image (step624) and the modified YIQ image is converted back into RGB (step 626)and output (step 628) as the post-processed image.

In the above-described example, determination of a relationship betweenmovement of the image capture device, time and blur extent is made onthe basis of a predetermined, transformation-based model relating theimage capture device movement, time and blur extent. However, onealternative to modeling the relationship is to capture test images usingvarious combinations of image capture device movement rate and exposuretime, and to measure blur extent in the test images. FIG. 14 is aflowchart showing the steps performed during this alternative technique100 a wherein test images for determining the relationship betweenmovement of the image capture device, time and blur extent are captured.

Referring to FIG. 14, a number of combinations of focal lengths, angularvelocities and exposure times are predetermined and used to configurethe image capture device to capture test images (step 150). FIG. 15shows an example of an array of test images taken at variouscombinations of exposure time and angular velocity for a particularfocal length of the image capture device. Once the test images have beencaptured, an extent of blur is measured in each of the test images (step152) and associated with respective ones of the combinations of focallength, angular velocity and exposure time. FIG. 16 is a graph showingrelationships between exposure time and blur extent measured in theimages of FIG. 15, for particular angular velocities. Combinations ofangular velocity and exposure time are then associated with ones ofpredetermined acceptable levels of blur extent (which may bepredetermined as a user-selectable range of discrete amounts by design)in order to relate angular velocity and exposure time for eachacceptable blur extent level (step 154). FIG. 17 is a graph showingrelationships between exposure time and angular velocity for variousacceptable blur extents. The relationships such as those shown in FIG.17 may be stored in the image capture device or elsewhere as discretevalues in a lookup table (step 156). Alternatively they may bemaintained as functions, depending on design considerations relating toprocessing and storage requirements, as would be understood.

In the first embodiment with reference to step 400 of FIG. 3,configuration of the image capture device with respect to maximumexposure time E_(max) and required image intensity is made on the basisthat the auto-exposure settings of the image capture device areover-ridden when the maximum exposure time E_(max) is exceeded by thebrightness-based automatic exposure time E_(auto). Alternatives however,are available. For example, one alternative method 400 a for configuringthe image capture device given a maximum exposure time E_(max), thatdoes not rely on a separate auto-exposure function will now be describedwith reference to FIG. 18.

First, a number of variables are retrieved. The first set of variablesis the minimum and maximum exposure times E_(device) _(—) _(min) andE_(device) _(—) _(max) supported by the image capture device (step 450).The second set of variables is the acceptable minimum and maximumintensity range settings y_(min) and y_(max), specified by a user orpredetermined by design (step 452). A third variable is theuser-specified exposure time step setting E_step (step 454).

A test image is then captured using a current exposure setting E_(true)of the image capture device (step 456). An average intensity of pixelsy_(measured) in the captured test image is then measured (step 458). IfE_(true) is greater than E_(max) (step 460), then E_(true) is re-set toequal E_(max) (step 462) because blur would otherwise occur that isgreater than the blur extent limit. The process at and following step456 is then re-iterated using the new E_(true).

If at step 460 E_(true) is not greater than E_(max), then E_(true) iscompared to E_(device) _(—) _(min) (step 464). Should E_(true) be lessthan E_(device) _(—) _(min), then the average intensity of pixelsy_(measured) is compared to the minimum intensity setting y_(min) (step466) to determine a re-set value of E_(true) for re-iterating theprocess at and following step 456. In order to determine the re-setvalue, should y_(measured) be greater than or equal to y_(min), thenE_(true) is set to E_(min) (step 470). On the other hand, shouldy_(measured) be less than y_(min), then E_(true) is re-set to the lowerof E_(true)+E_step, and E_(max) (step 468).

If, at step 464, E_(true) is not less than E_(device) _(—) _(min), theny_(measured) is compared to y_(max) (step 472). If y_(measured) isgreater than y_(max), then E_(true) is re-set to the greater ofE_(true)−E_step, and E_(min) (step 474) in order to bring the intensitycloser to the acceptable range, and the process at and following step456 is then re-iterated using the re-set E_(true). If y_(measured) isnot greater than y_(max), then y_(measured) is compared to y_(min) (step476). If y_(measured) is also greater than or equal to y_(min), thenboth exposure time and intensity are within acceptable ranges, and theimage may be captured using E_(true) (step 482). On the other hand, ify_(measured) is less than y_(min), then should E_(true) be equal toE_(max) (step 478) no improving adjustments are possible, and the imagemay be captured using E_(true) (step 482). Otherwise, E_(true) is set tothe lower of E_(true)+E_step, and E_(max) (step 480) and the process isre-iterated at step 456.

The process shown by flowchart in FIG. 18 may be better understood withreference to the graph of image intensity vs. exposure time illustratedin FIG. 19. The graph is sectioned into regions A-F, and eachcombination of E_(true) and y_(measured) falls into one region. Theregion into which a combination falls dictates the re-set value ofexposure time, with the arrows representing the rules schematically:

A: E_(true)=min(E_(true)+E_step, E_(max))

B: E_(true)=ex_(min)

C: E_(true)=min(E_(true)+E_step, E_(max))

D: E_(true) not changed; capture image

E: E_(true)=max(E_(true)−E_step, E_(min))

F: F_(true)=E_(max)

As will be understood, the operation of this alternative method 400 a issimilar to the operation of a standard auto-exposure function. That is,in regions C and E, exposure is adjusted based on image brightness.However, the method differs from typical auto-exposure systems in regionF, where exposure is reduced regardless of image brightness.

Processing the captured image and in particular brightening the imagewhere required has been described herein with reference to step 600 onthe basis of a particular saturation model. Again, alternatives areavailable. For example, one alternative method 600 a for brightening acaptured image relies on shifting and stretching its histogram of pixelintensities as will now be described with reference to FIG. 20.

First, user pre-defined (i.e. adjustable) parameters max_y_(max) andmin_y_(max) intensity values, and the min_n_(dim) pixel percentage valueare retrieved (step 650).

The max_y_(max) intensity value is a histogram stretch upper threshold.A value y_(max) is calculated so that most of the information in theimage (i.e., pixels having an intensity within two (2) standarddeviations of the mean intensity) has an intensity less than thisy_(max). The histogram will be stretched so that pixels withintensity>=y_(max) are stretched to intensity 255. If y_(max) is closeenough to 255 already, (that is, if y_(max)>=max_y_(max)), then theimage need not be further brightened—it is bright enough.

The min_y_(max) value is a histogram stretch lower threshold. If y_(max)is less than this value, the image appears very dim. Because stretchingthe histogram from the y_(max) intensity would lead to an unnaturallooking image, the histogram is stretched from the min_y_(max)intensity.

An n_(dim) value represents the percentage of pixels that will bestretched and shifted normally. The remaining 1−n_(dim) pixels aresaturated at 255. In order to minimize loss of detail, the brighteningprocess does not attempt to brighten the image if n_(dim) is less thanmin_n_(dim). Otherwise, too many pixels would be saturated.

During testing, the following values were employed:

max_y_(max)=204;

min_y_(max)=128; and

min_n_(dim)=80%.

Once the predefined settings have been retrieved, the captured image isconverted to YIQ space, and the intensity channel Y obtained (step 652).A histogram of pixel intensity accumulation is then created from the Ychannel by grouping pixels according to their Y channel values (step654).

Mean intensity (μ) and standard deviation (ρ) values are calculated fromthe histogram (step 656), and the y_(max) value is set (step 658) asfollows:y _(max)=min(μ+2ρ,min_(—) y _(max))  (68)

Then, the number of pixels with intensity lower than y_(max) is counted(step 660) and used to obtain a percentage of pixels n_(dim) withintensity lower than y_(max). In the event that the following conditionis not true (step 662):y _(max)<=max_(—) y _(max) and n _(dim)>=min_(—) n _(dim)   (69)

then there is no need to brighten the captured image, and theunprocessed captured image is output (step 664). If the condition atstep 662 is true, then the histogram stretch factor is defined (step666) as follows:stretch=(128+μ/y _(max))/y _(max)   (70)

Similarly, the histogram shift factor is defined (step 668) as follows:shift=128−stretch*μ  (71)

Using Equations (70) and (71), each pixel's intensity y_(ij) is thenadjusted (step 670) as follows to create a modified intensity channel Y:y _(ij)=(y _(ij)*stretch)+shift   (72)

The modified intensity channel Y replaces the one extracted from the YIQimage (step 672) and the modified YIQ image is converted back into RGB(step 674) and output (step 676) as the post-processed image.

FIG. 21 is an array of images and corresponding pixel intensityhistograms. An unprocessed captured image is shown with its histogram,and then processed images with first stretched, and then shiftedhistograms are shown. For comparison, an image taken using standardauto-exposure without limiting exposure time, and its correspondinghistogram is also shown.

Although embodiments have been described, those of skill in the art willappreciate that variations and modifications may be made withoutdeparting from the spirit and scope thereof as defined by the appendedclaims.

1. A method of determining a maximum exposure time to limit motion blurduring capture of an image with an image capture device, comprising:determining extents of image blur that result from respectivecombinations of movement rate and exposure time of the image capturedevice; in accordance with a predetermined blur extent limit and thecombinations, deriving a relationship between movement rate and exposuretime of the image capture device; measuring a movement rate of the imagecapture device; and based on the measured movement rate and therelationship, obtaining the maximum exposure time; and wherein thedetermining comprises: defining a model of correlation between movementof the image capture device relative to a subject and resultant blurextent in pixels; and calculating blur extents resulting from rates ofmovement of the image capture device over time using the model.
 2. Themethod of claim 1, wherein the determining comprises: capturing aplurality of images using respective combinations of movement rate andexposure time of the image capture device; and measuring a blur extentin each of the captured images.
 3. The method of claim 1, wherein themodel of correlation is based on a simple lens model.
 4. The method ofclaim 3, wherein the model of correlation is also based on purerotation.
 5. The method of claim 3, wherein the model of correlationcomprises: a first transformation from world object coordinates to imagecapture device coordinates; a second transformation from image capturedevice coordinates to image plane coordinates; a third transformationfrom image plane coordinates to focal plane coordinates; and a fourthtransformation from focal plane coordinates to image pixel coordinates.6. The method of claim 5, wherein the first transformation is arotation.
 7. The method of claim 5, wherein the second transformation isa perspective projection.
 8. The method of claim 5, wherein the thirdtransformation is a scaling.
 9. The method of claim 5, wherein thefourth transformation is a scaling.
 10. The method of claim 1, whereinthe calculating comprises: for each combination of movement rate andexposure time, obtaining a respective field of motion vectors for pixelsin each of the X, Y and Z directions; for each field, determining anaverage motion vector; and calculating the magnitude of each averagemotion vector as a blur extent.
 11. The method of claim 10, wherein eachaverage motion vector is determined from a subset of the motion vectorsin a respective field.
 12. The method of claim 11, wherein the subsetcomprises nine motion vectors.
 13. The method of claim 12, wherein thenine motion vectors are the left-top, middle-top, right-top,left-center, middle-center, right-center, left-bottom, middle-bottom andright-bottom vectors in a field.
 14. The method of claim 10, whereineach average motion vector is determined by weighting motion vectors ina field in accordance with a respective proximity to the field center.15. The method of claim 1, wherein the determined extents of image blurthat result from respective combinations of movement rate and exposuretime of the image capture device are stored in a table for each of theX, Y and Z directions.
 16. The method of claim 15, wherein therelationship between movement rate and exposure time of the imagecapture device is determined by obtaining the greatest exposure time ineach table for each movement rate that would result in a blur extentbeing no greater than the predetermined blur extent limit.
 17. Themethod of claim 16, wherein movement rates in each of the X, Y and Zdirections are indices in an exposure limit lookup table.
 18. The methodof claim 17, wherein the indices are nonlinearly spaced to provide ahigher proportion of table entries at lower movement rates.
 19. Themethod of claim 1, wherein the movement rate is angular velocity.
 20. Amethod of capturing an image with an image capture device, comprising:determining a movement rate of the image capture device; on the basis ofthe movement rate and a blur extent limit, obtaining a maximum exposuretime from a predetermined relationship between movement rate andexposure time for the image capture device; exposing an image sensor ofthe image capture device to capture an image, the exposing being limitedin duration by the maximum exposure time; automatically adjusting atleast one of an aperture size of the image capture device and asensitivity of the image sensor to thereby effect exposure level; andpg,37 in the event that a measure of intensity of the captured image isless than a predetermined threshold, automatically increasing theintensity value of at least a portion of pixels in the captured image.21. An apparatus for determining a maximum exposure time to limit motionblur during capture of an image with an image capture device,comprising: a processor determining extents of image blur that resultfrom respective combinations of movement rate and exposure time of theimage capture device and, in accordance with a predetermined blur extentlimit and the determined combinations, deriving a relationship betweenmovement rate and exposure time of the image capture device; a storagedevice storing the relationship; a measuring device that measures amovement rate of the image capture device; and a lookup moduledetermining from the relationship stored on the storage device themaximum exposure time based on the measured movement rate and thepredetermined blur extent limit.
 22. The apparatus of claim 21, whereinthe processor performs the determining by: capturing a plurality ofimages using respective combinations of movement rate and exposure timeof the image capture device; and measuring a blur extent in each of thecaptured images.
 23. The apparatus of claim 21, wherein the processorperforms the determining by: defining a model relating movement of theimage capture device relative to a subject and resultant blur extent inpixels; and calculating blur extents resulting from rates of movement ofthe image capture device over time using the model.
 24. The apparatus ofclaim 21, wherein the measuring device is a gyro.